Context diary application for a mobile terminal

ABSTRACT

The invention provides a presentation of contextual information to a digital device in relation to a period of time. As such the invention provides for efficient user access to historical, as well as, current and future or forecasted contextual information. In addition, the present invention provides for current contextual information to be presented in an idle mode view, a scrolling bar mode view or another inconspicuous view. Additionally the present invention provides a novel means of using context information to create multimedia file presentations; image or video file presentations that can be associated with previous periods of time, locations or other context data.

RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.11/026,663, filed Dec. 31, 2004, the entirety of which is incorporationherein by reference.

BACKGROUND OF THE INVENTION

Mobile digital terminals, such as mobile telephones and the like arebecoming increasing commonplace throughout the world. Recenttechnological advances in processing and storage capacity have made itpossible for such terminals to become multi-functional. For example, nolonger is the mobile telephone limited to telephonic voice communicationbut may also provide other means of digital communication, such asdigital networking (i.e., Internet communication, text messaging, andthe like), image or video capture means, digital video broadcastreception and the like.

As the communication means and the functionality of such terminalsexpands, such terminals will readily come in contact with anincreasingly voluminous amount of media items, such as email, voicemail, digital image files, digital video files and the like. As thememory capacity of theses terminals increases, either via internalmemory capacity or the ability to readily access external memory storageelements, the user of the device will have the capability to storeand/or access all of these media items.

The increasing amount of media items stored or available to the digitaldevice has been the impetus behind implementation of file storage andorganization applications and more specifically, calendar-typeapplications that allow the user to categorize media items according tothe date and/or time associated with the media item. For example, theMicrosoft Outlook application, available form the Microsoft Corporationof Redmond, Wash., provides for storage and categorization of email,voicemail and the like according to the date of receipt or the date oftransmission. Other applications have been developed that are moresuitable to mobile terminals and provide for the user to efficientlylocate a media item according to the time associated with the mediaitem. For example, if the media item is an image file captured by amobile terminal (i.e., a photograph taken by a camera equipped mobiletelephone) the calendar application will typically associate the imagefile with the date and time at which the image file was created or anevent associated with the image file (e.g., a birthday party, a meeting,a family event, etc). Thus, the calendar-applications allow the user tostore media files, such as email, short message service (SMS) messages,voicemail, image files, video files and the like in diary format toallow the user ready and efficient access to the media items. However,currently, extensive use of such mobile terminal calendar-typeapplications has not been realized by the device consumer.

Recent innovations have made it possible for the mobile terminal toprovide context-awareness functionality, such as location of the device,environment of the device's location, movement of the device and thelike. Such functionality is provided by including corresponding sensorsin the device or providing for the terminal to communicate with externalresources that provide context information. For example, the mobileterminal may be equipped with a Global Positioning Sensor (GPS) thatprovides the present location of the mobile device, a temperaturesensor, an altitude sensor, motion or acceleration sensors and the like.External resources may be any external device capable of providingcontextual information, such as access points, network nodes, othermobile communication devices and the like.

Context-awareness is a key area of potential for mobile devices becauseit enables differentiation from stationary devices, like desktopcomputers, because mobile use is characterized by constant change incontext (i.e., the device moves constantly and encounters constantlychanging environments). Moreover context-awareness enables an overallricher user-experience. It provides the user with valuable informationrelated to the context of the device.

In a typical context-awareness application the mobile terminal will becapable of providing current contextual information. For example, anapplication may implemented whereby the device is capable of providingthe user through a user interface, such as a display, the currentlocation of the device, the temperature, altitude, barometric pressureor the like of the device's current location.

Contextual information is also captured, continuously or at a given timeinterval and provided to certain types of applications that utilizecontextual information accumulated over time. For instance, in order toprovide contextually sensitive set of bookmarks according to thelocation, the application may need to record the browsing behaviors inrelation to the device location over time. However, the applicationsthat customize the behavior, i.e., context and the content set accordingto recognized usage patterns do need to display the raw context data tothe user. As such, the users are not aware that the context informationis being captured and no further utilization of the context data by theuser is provided for.

Therefore a need exists to provide a user access to captured contextdata and, more specifically, access to the captured context data inrelation to a period or moment in time. Such an application will providethe user of the mobile terminal ready and efficient access to capturedcontext data. This type of data will provide the user with a diary as towhere the device (and thus the user) was located at a previous point intime and other previous data related to the context of the terminal. Anadditional need exists to provide the user with various access modes tothe context data such that the data can benefit from the data and usethe data in other applications.

BRIEF SUMMARY OF THE INVENTION

The present invention provides for methods, computer program productsand devices for providing the user of digital device a diary of contextinformation. The diary provides context information, such as devicelocation, device movement, device environment and the like in relationto a period of time. In this regard, the diary utilizes a time-linebased user-interface to display captured context information. It shouldbe noted that for the purposes of the invention the term “a period oftime” includes a moment in time, an event in time and any other timeentity. As such, the diary provides a device user efficient access tocontextual information that is captured and stored by the digitaldevice. Thus, the invention provides a means of displaying raw contextinformation without the need to perform complicated analysis on theinformation prior to display. By limiting complicated analysis, theinvention does not require additional processing/computational power nordoes it require additional memory. In addition the diary provides theuser graphical presentations that allow for the user to navigate betweena general overview of contextual information to more detailed views ofselected context data.

In one embodiment of the invention a computer program product forproviding access to context information on a digital device is provided.The product includes a computer readable storage medium havingcomputer-readable program instructions embodied in the medium. Thecomputer-readable program instructions include first instructions forgenerating a context view that provides for at least one category of thecontext information and provides user-access to at least one item ofcontext information within the at least one category; and secondinstructions for generating a timeline view that is presented incombination with the context view and provides user-access to the atleast one item of context information in relation to a period of time.In this regard the product is a context diary that provides the userwith access to context information in relation to a period of time,moment in time, event, location or some other contextual parameter.

The product may also include third instructions for generating a mediaview that provides access to at least one digital media file and ispresented in combination with the timeline view to provide user-accessto the at least one digital media file in relation to a period of time.While the context diary may be a stand-alone application in manyinstances it will be combined with a calendar application or a mediadiary application to provide one concise application that providesaccess to future information, current information and past information.In the combined context, media view product the context information willtypically be associated with one or more digital media files. Theassociation will typically be related to a time period and will indicatewithin the respective context and media views by some form of indicator.The first, second and third instructions will typically be implementedconcurrently to generate a context access view that provides user-accessto the context view in combination with the timeline view and the mediaview in combination with the timeline view.

The invention is also embodied in a method for accessing contextinformation on a mobile terminal. The method includes the steps ofmonitoring the mobile terminal environment to determine one or morecontext data and presenting, on a mobile terminal display, a contextview that provides at least one context category associated with the oneor more context data and at least one context item within the at leastone context category. The at least one context item is related withinthe context view to a period of time, typically a past period of time.The step of monitoring of the mobile terminal environment to determineone or more context data further may include receiving one or morecontext data from within the mobile terminal, i.e., an internal sensoror an internal application, or receiving one or more context data froman external context data source that is in wireless communication withthe mobile terminal; i.e., an Internet resource, a device, a sensor orbroadcast data, etc. The step of presenting a context view may furtherinclude presenting, on the mobile terminal display, the context view incombination with a timeline view that provides at least one contextcategory associated with the one or more context data and at least onecontext item within the at least one context category associated with aperiod of time within the timeline view. The context category mayinclude calendar events, terminal location, short-range communicationdevices encountered, location characteristics, terminal movement,individuals encountered, personal user health characteristics,environment characteristics and the like.

The method may further include the steps of presenting, on the mobileterminal display, a calendar view; and electing a period of time withinthe calendar view for presenting the context view that is associatedwith the period of time.

The invention is also defined by a mobile terminal device for providinguser-access to context data. The device includes a display, a processingunit in communication with the display and a memory module incommunication with the processor. The memory module stores a contextdiary application executed by the processor. The context diaryapplication generates a context view and a timeline view that aredisplayed on the display concurrently such that the user may be providedaccess to context data within the context view in relation to a periodof time within the timeline view.

The device will typically include one or more sensors in communicationwith the processor for sensing one or more personal context data. Theone or more context data is provided to the context diary applicationsuch that the context view provides user-access to the context datasensed by the one or more sensors. The memory module may further storeone or more context data-generating applications that generate one ormore context data. The one or more context data is provided to thecontext diary application such that the context view providesuser-access to the context data generated by the one or more contextdata-generating applications.

The device may also additionally include a wireless communication modulein communication with the processor that provides for wirelesscommunication with external context sources that provide one or morecontext data to the application such that the context view providesuser-access to the context data communicated from the external elements.

In yet another embodiment of the present invention a user interfaceapplication for a digital terminal is defined. The application includesfirst instructions for generating the display of a context view incombination with a timeline view that provides a display of one or morecontext data in relation to periods of time and second instructions forselecting, by user input, one of the one or more context data to providea more detailed context view related to the data associated with theactivated context data. In this regard, the user interface applicationprovides the user with a context view that provides a general overviewof context data and a user may select an entry from within the view toprovide for the presentation of more detailed information related to theentry.

Additionally the user interface may provide for a generation and displayof a media view in combination with the timeline view that provides adisplay of one or more media files in relation to periods of time. Thefirst instructions may additionally provide for visual indication on thedisplay of an association between one or more context data and the oneor more media files. In this regard, the second instructions may furtherinclude instructions for selecting, by user input, the one or morecontext data or the one or more media files to visually indicate anassociation between one or more context data and one or more mediafiles.

The invention is also embodied in a method for providing user-access tocurrent context information on a mobile terminal. The method includesthe steps of monitoring the current mobile terminal environment todetermine one or more context data, presenting the currently monitoredcontext data in user-accessible form on a display of the mobileterminal, updating the context data on the display when there is achange in the mobile terminal environment; and accessing the monitoredcontext data to provide user management of the context data.

The step of monitoring the mobile terminal environment to determine oneor more context data may further include receiving context data fromwithin the mobile terminal or receive the context data from an externalcontext source that is in wireless communication with the mobileterminal.

The step the presenting the currently monitored context data inuser-accessible form on a display of the mobile terminal monitoredcontext data in user-accessible form on a display of the mobile terminalmay further include presenting the monitored context data on an idlescreen mode, a ticker bar mode that scrolls monitored context data, on abackground/wallpaper image, within a secondary application or the like.

The step of updating the context data on the display when there is achange in the mobile terminal environment may further defines the changein the mobile terminal environment as a change in terminal location;detection of an external context data source; acknowledging thedisappearance of a signal from a detected external context data sourceand detecting a new value for at least one of the monitored contextdata.

Thus, the present invention provides contextual information to digitaldevice user in a presentation that associates the contextual informationwith a period of time. As such the present invention provides forefficient user access to historical, as well as, current contextualinformation. In addition, the present invention provides for currentcontextual information to be presented in an idle mode view, a scrollingbar mode view or another inconspicuous view. Additionally the presentinvention provides a novel means of using context information to createmultimedia file presentations; image or video file presentations thatcan be associated with previous periods of time, locations or othercontext data.

BRIEF DESCRIPTION OF THE DRAWINGS

Having thus described the invention in general terms, reference will nowbe made to the accompanying drawings, which are not necessarily drawn toscale, and wherein:

FIG. 1 is a cell diagram of the overall system for context informationdistribution and storage, in accordance with an embodiment of thepresent invention.

FIG. 2 is a cell diagram of a mobile terminal implementing a contextdiary application in accordance with an embodiment of the presentinvention.

FIG. 3 is a cell diagram of a mobile terminal implementing a contextengine for capture and processing of context data, in accordance with anembodiment of the present invention.

FIG. 4 is a screenshot or view of the context diary application thatcombines the context view with the timeline view, in accordance with anembodiment of the present invention.

FIG. 5 is a screenshot or view of the diary application that combinesthe context view, the media view and the timeline view, in accordancewith an embodiment of the present invention.

FIG. 6 is an alternate screenshot or view of the diary application thatcombines the context view, the media view and the timeline view, inaccordance with an embodiment of the present invention.

FIG. 7 is a screenshot or view of the context diary application in whichthe cells of the context view matrix provide for context information, inaccordance with an embodiment of the present invention.

FIG. 8 is a screenshot or view of the context diary application in whichthe cells of the context view matrix provide for visual attributes/iconsthat are associated with context information, in accordance with anembodiment of the present invention.

FIG. 9 is an idle screen mode view of a diary application that providesa presentation of current context information, in accordance with anembodiment of the present invention.

FIG. 10 is a rolling ticker bar window view of a diary application thatprovides a presentation of current context information, in accordancewith an embodiment of the present invention.

FIGS. 11A-11G provide various views encountered by a device user increating a multimedia presentation within the diary application, inaccordance with an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present inventions now will be described more fully hereinafter withreference to the accompanying drawings, in which some, but not allembodiments of the invention are shown. Indeed, these inventions may beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided so that this disclosure will satisfy applicable legalrequirements. Like numbers refer to like elements throughout.

The present invention provides for methods, computer program productsand devices for providing the user of digital device a contextinformation management application, such as, a diary of contextinformation. The diary provides context information, such as devicelocation, device movement, device environment and the like in relationto a period of time. In this regard, the diary utilizes a time-linebased user-interface to display captured context information. It shouldbe noted that for the purposes of the invention the term “a period oftime” includes a moment in time, an event in time and any other timeentity. As such, the diary provides a device user efficient access tocontextual information that is captured and stored by the digitaldevice. In addition the diary provides the user graphical presentationsthat allow for the user to navigate between a general overview ofcontextual information to more detailed views of selected context data.

The context diary of the present invention may be provided for in aseparate stand-alone application or the context diary may beincorporated in existing calendar-type applications that display atimeline representation of future events. Typically, calendar-typeapplications are implemented on digital devices 10, in FIG. 1, such asmobile telephones, personal digital assistants (PDAs), mobilecommunication devices, personal computers, laptop computers and the liketo provide the user a planning mechanism for future events. Inembodiments of the invention in which the context diary is incorporatedin a calendar-type application the captured context information historywill be displayed in relation to a historical time line. In anotherembodiment, the context diary may incorporate a media diary or mediamanagement application/calendar application or vice versa.

FIG. 1 provides a general overview of a system for capturing contextinformation, in accordance with an embodiment of the present invention.The digital device 10, typically a mobile digital communication devicewill typically include a central processing unit 12, user interfaces 14,such as a keypad, display and the like, wireless communicationinterfaces 16, such as transceivers and the like, and memory elements 18that store various device related applications 20, including the contextdiary application of the present invention alone or incorporated withapplications or features of the media diary and/or the calendarapplication.

The digital device 10 will either rely on personal sources, such aspersonal 1 sensors and personal applications for context and/or contentinformation and/or the device may rely on external sources thatcommunicate with the device via a network. Personal sensors aretypically internal to the digital device that is operating the contextdiary. However, in certain embodiments of the invention, the personalsensors may be separate from the digital device and in communicationwith the digital device through an acceptable communication interface.If the device relies on personal sources for contextual information,these sources may include, but are not limited to, personal sensors 30,personal applications 32, personal databases 34 and the like. Thesensors 30 may take the form of Global Positioning Sensors (GPS),temperature sensors, altitude sensors, barometric pressure sensors,motion sensors, acceleration sensors, electronic compass, medicalsensors, such as blood pressure sensors, blood sugar sensors, heart beatsensors and the like. One or more of the personal sensors 30 may beincorporated in a personal wrist watch or in a personal health ormedical monitoring device. The personal applications 32 may include acalendar application, a media diary application or the like and thepersonal databases 34 may include a media storage application, an eventlog, a file manager or the like.

If the device relies on external sources for contextual information thedevice will typically monitor the devices environment for any contextand or content information from external sources. Monitoring theenvironment may involve one or several wired or wireless communicationmeans, including, but not limited to, Bluetooth® (communication in theabout 2.45 GHz frequency range), Ultra WideBand (UWB) communication,Wireless Local-Area Network (WLAN), Radio frequency Identification(RFID), wireless telecommunication, Internet, Digital Video Broadcast(DVB), Digital Multimedia Broadcast (DMB), Amplitude Modulation (AM),Radio Data System (RDS), IrDa, Digital Audio Broadcasting (DAB) and thelike. The context and/or content information may be registered orupdated within the context diary and/or the content diary when detectedor received or it may be registered or updated based on predefined timeintervals, such as every second, minute or hour.

The external sources include any external electronic devices that havethe appropriate communication interface to communicate with the digitaldevice. The external sources may include, but are not limited to,devices 40, sensors 42, Internet resources 44, digital broadcastinformation 46 and the like. The devices 40 may include printers,display devices, access points, network nodes, industrial controldevices, domestic appliances, consumer electronic devices, computers,mobile communication devices, wrist watches, etc. The devices 40 mayinclude one or more sensors and/or applications. The sensors 42 mayinclude Global Positioning Sensors (GPS), temperature sensors, altitudesensors, barometric pressure sensors, motion sensors, accelerationsensors and the like. The Internet resources 44 may be any contentavailable through the Internet and the digital broadcast information 46may be any information transmitted via a wireless or wireline broadcastnetwork. For example, the digital broadcast information may include anelectronic program guide (EPG), an electronic service guide (ESG) or thelike. In addition to the external sources shown in FIG. 1, the sourcesmay be any received content, such as television or radio broadcast,video or audio files, games, content delivery via the Internet, wirelesstelecommunication network content and the like. Context data receivedfrom external sources may also include access information, userinterface information and/or settings information. For example, aprinter device may provide its access information, a room temperaturecontrol unit may provide its setting information and instructions onchanging the settings. The digital device may store this data and thediary application will access this information when the device comes incontact with external source. Thus, a user may be provided access andsettings to the external source through the context diary.

In addition to the digital device and the external sources, the systemmay additionally include storage means in the form of external massstorage units that store either a portion of or the entirety of thedigital device's context information, content information and/orcalendar information. Examples of such storage units include, but arenot limited to, network servers 50, a personal computer (PC) 52 orlaptop computer or a portable and/or wireless personal server device 54,such as a portable music/video/multimedia player that are accessible viaa wireless or wireline communication network. In addition to storingcontext information, the storage means may also store and run thecontext diary application of the present invention and communicate theuser interface aspects of the application to the digital device. Storageand execution of the context diary application at the storage means istypically implemented in instances in which the digital device haslimited memory capacity or processing capabilities.

Reference is now made to FIG. 2, which illustrates one type of digitaldevice 10, a mobile terminal that would benefit from embodiments of thepresent invention. It should be understood, however, that the mobileterminal illustrated and hereinafter described is merely illustrative ofone type of terminal that would benefit from the present invention and,therefore, should not be taken to limit the scope of the presentinvention. While several embodiments of the terminal are illustrated andwill be hereinafter described for purposes of example, other types ofterminals, such as portable digital assistants (PDAs), pagers, laptopcomputers, mobile telephones, mobile communication devices, wristwatches, digital cameras, digital camcorders, digital audio devices,digital video devices, wireless personal server devices and other typesof devices or any combination of the mentioned electronic systems ordevices, can readily employ the present invention.

As shown, in addition to an antenna 60, the mobile terminal includes atransmitter 62, a receiver 64, a controller 66, and a processor 68 thatprovides signals to and receives signals from the transmitter andreceiver, respectively. These signals include signaling information inaccordance with the air interface standard of the applicable cellularsystem, and also user speech and/or user generated data. In this regard,the mobile terminal can be capable of operating with one or more airinterface standards, communication protocols, modulation types, andaccess types. In some embodiments of the mobile terminal of the presentinvention, the processor 68 includes the controller 66.

It is understood that the controller 66 includes the circuitry requiredfor implementing the audio and logic functions of the mobile terminal.For example, the controller may be comprised of a digital signalprocessor device, a microprocessor device, and various analog-to-digitalconverters, digital-to-analog converters, and other support circuits.The control and signal processing functions of the mobile terminal areallocated between these devices according to their respectivecapabilities. The controller can additionally include an internal voicecoder (VC) 70, and may include an internal data modem (DM) 72. Further,the controller may include the functionally to operate one or moresoftware programs, which may be stored in memory (described below). Forexample, the controller may be capable of operating a connectivityprogram, such as a conventional Web browser. The connectivity programmay then allow the mobile terminal to transmit and receive Web content,such as according to the Hypertext Transfer Protocol (HTTP) and/or theWireless Application Protocol (WAP), for example.

The mobile terminal also comprises a user interface including aconventional earphone or speaker 74, a ringer 76, a microphone 78, adisplay 80, and a user input interface, all of which are coupled to thecontroller 66. The user input interface, which allows the mobileterminal to receive data, can comprise any of a number of devicesallowing the mobile terminal to receive data, such as a keypad 82, atouch display (not shown) or other input device. In embodimentsincluding a keypad, the keypad includes the conventional numeric (0-9)and related keys (#, *), and other keys used for operating the mobileterminal.

As shown in FIG. 2, the mobile terminal can include a radio frequency(RF) transceiver 84 (which includes a RFID reader) so that data can beshared with and/or obtained from electronic devices including otherradio frequency transceivers or transponder tags. In some embodiments ofthe mobile terminal of the present invention, the RFID reader alsoincludes the processor 68 and/or the controller 66. The mobile terminalcan additionally, or alternatively, include other short-rangecommunication transceivers, such as, for example, an infrared (IR)transceiver 86, and/or a Bluetooth® (BT) transceiver 88 (operating inthe 2.45 GHz frequency band) operating using Bluetooth brand wirelesstechnology developed by the Bluetooth Special Interest Group. The RFtransceiver 84 and/or other transceivers may be associated with themobile terminal in any manner known to those skilled in the art. Forexample, in some embodiments, the RF transceiver 84 and/or othertransceivers may be integrated in the mobile terminal or may be separatefrom, but in communication with, the mobile terminal, such as via anytype of wireline and/or wireless techniques. The mobile terminal cantherefore additionally or alternatively be capable of transmitting datato and/or receiving data from electronic devices and/or tags. Althoughnot shown, the mobile terminal may additionally or alternatively becapable of transmitting and/or receiving data from electronic devicesand/or tags according to a number of different wireless networkingtechniques, including, but not limited to, for example, WLAN techniquessuch as IEEE 802.11 techniques, wireless telecommunication techniquessuch as GSM, UTMS or 3G or the like.

The mobile terminal also includes a battery 90, such as a vibratingbattery pack, for powering the various circuits that are required tooperate the mobile terminal, as well as optionally providing mechanicalvibration as a detectable output. For example, the battery 90 mayprovide at least a portion of the power required to operate the RFIDreader. The mobile terminal can further include memory, such as asubscriber identity module (SIM) 92, a removable user identity module(R-UIM) or the like, which typically stores information elements relatedto a mobile subscriber. In addition to the SIM, the mobile terminal caninclude other removable and/or fixed memory. In this regard, the mobileterminal can include volatile memory 94, such as volatile Random AccessMemory (RAM) including a cache area for the temporary storage of data.The mobile terminal can also include other non-volatile memory 96, whichcan be embedded and/or may be removable. The non-volatile memory canadditionally or alternatively comprise an EEPROM, flash memory or thelike. The memories can store any of a number of pieces of information,and data, used by the mobile terminal to implement the functions of themobile terminal. The memories can also store one or more applicationscapable of operating on the mobile terminal.

As a means of capturing context information, the mobile terminal mayinclude a positioning sensor, such as a global positioning system (GPS)sensor 98. In this regard, the GPS sensor is capable of determining alocation of the mobile terminal, such as longitudinal and latitudinaldirections of the mobile terminal. The mobile terminal may also includeone or more additional sensors 100 for sensing the ambient conditions ofthe mobile user and, more particularly, the mobile terminal operated by,or otherwise under the control of, the mobile user. In this regard, themobile terminal may include sensors such as, for example, a positioningsensor, a touch sensor, an audio sensor, a compass sensor, an ambientlight sensor, an ambient temperature sensor, an ambient pressure sensor,a proximity sensor, and/or an acceleration sensor.

The audio sensor can include the microphone 78, for example, which candetect speech or environmental sounds. The positioning sensor caninclude, for example, the GPS sensor 98. Additionally, or alternatively,the positioning sensor can include, for example, a radio beacontriangulation sensor that determines the location of the mobile terminalby means of a network of radio beacons, base stations, or access points,as is described for example, in Nokia European patent EP 0 767 594 A3,entitled “Mobile Station Positioning System” published on May 12, 1999,the contents of which are hereby incorporated by reference in itsentirety. As will be appreciated, the sensors can also be located inaccessory-like mobile terminal covers and/or in a wireless accessorysuch as a Bluetooth®-enabled device. The sensors may further be locatedin the environment such as in the user's rooms or vehicles. Also,information capable of being measured by the mobile terminal, such asthe time duration of use of the mobile terminal, can be received assensor data by the mobile terminal. For more information on suchsensors, see U.S. patent application Ser. No. 09/854,628, entitled“Context Sensitive Web Services” filed May 15, 2001 and published Nov.21, 2002 as U.S. Patent Application Publication No. 2002/0173295, thecontents of which are hereby incorporated by reference in its entirety.

The measured and/or sensed data may then be utilized, such as by theprocessor 66, to determine a context and/or change in context of themobile terminal. Although the context and/or a change in context may bedetermined in various manners, the measured and/or sensed data may betransmitted according to one exemplary embodiment to a centralblackboard that delegates the determined changes in the context todifferent destinations, such as the RFID transceiver 84, which includesa RFID reader, utilizing a context exchange architecture.

FIG. 3 illustrates a context exchange architecture of a mobile terminal10 to determine a context and/or change in context of the mobileterminal in accordance with one embodiment of the present invention. Asshown in FIG. 3, the context exchange architecture of the mobileterminal 10 according to one embodiment of the present inventionincludes one or more sensors 100 that are capable of providing measuredand/or sensed data, as described above. The sensors are further capableof preprocessing the measured and/or sensed data into “context atoms,”such as by a sensor application program interface (API), and thereafterstored. As used herein, a “context atom” generally refers to a specificpiece of context-related information. A context atom can comprise anentity that typically has an associated name, value, source (i.e.,originator), date and/or any of a number of other attributes. Moreparticularly, the context atom can include other attributes includingmetadata information, such as the reliability of the context-relatedinformation, as well as its accuracy, source, time of creation, etc. Forexample, a temperature sensor can measure the ambient temperature of theenvironment around the mobile terminal 10, and thereafter process thetemperature into a context atom. Also, for example, the mobile terminalmay store a directory that can measure, or record, the last call made orreceived in the mobile terminal. The last call data can then beprocessed into a context atom. Further, the terminal may store metadatainformation on actions and manipulations, such as editing, sending,renaming displaying, etc., performed on a data file. This informationcan then be processed into a context atom. Further, for example, athree-axis acceleration sensor can measure movement of the mobileterminal. Thereafter, the movement data can be processed into a contextatom that may include the current absolute value of acceleration, thevalue integrated over several seconds, and/or a rough estimate whetherthe mobile terminal, or more particularly the mobile terminal user, iswalking, running, standing still or perhaps sitting in a car.

The context exchange architecture of the mobile terminal 10 alsoincludes a context engine 102, which is capable of receiving, andthereafter storing, the context atoms, as well as providing access tothe context atoms in accordance with embodiments of the presentinvention. The context engine includes a context silo 104 and a contextAPI 106. The context silo can, in turn, include a blackboard 108 and asubscription manager 110. The blackboard is a storage unit forcontext-related information and, as such, typically stores the contextatoms received from the sensors or other context sources (such as otherelectronic devices, applications, applications residing behind anetwork, other user terminals, etc.). Additionally, the blackboard canprovide an API to query and set context atoms. The subscription manager,in turn, can provide subscriptions to various of the context atomsand/or notifications when such context atoms change. The subscriptionmanager can therefore periodically poll the context atoms based uponsuch subscriptions and notifications. To provide access to the contextsilo, the context engine further includes the context API. In thisregard, the context API allows one or more user applications 112 and/orone or more timers 114 to access the context data from the blackboard,in accordance with embodiments of the present invention. The timer 114generally tracks the time between determinations of a change in contextof the mobile terminal 10. In other embodiments of the mobile terminal10, the timer 114 may be included in the user application(s) 112 and/orthe context engine 102.

The blackboard 108 is capable of storing context atoms for an indefiniteperiod of time, unless otherwise instructed. For example, one or morecontext atoms can include an expiration time such that after therespective context atoms expire, such context atoms can be deleted fromthe blackboard, with a notification transmitted to devices subscribing(described below) to the respective context atom, if so desired. As willbe appreciated, however, each time a context atom is updated withdifferent information (e.g., different values), the expiration time canbe reset. Also, in such instances as when a new context atom is storedin the blackboard that has the same name and context source (indicatingthat it appeared from the same source) as a previous context atomalready in the blackboard, the new context atom can replace the previouscontext atom, which can then be either discarded or moved to a historybuffer, for example.

The context exchange architecture further includes a data store 116coupled to the context engine 102. The data store is capable of storinginformation such as, for example, user preferences, profile information,as well as permissions for a privacy engine 118, also included withinthe architecture. The privacy engine is capable of providing securityand privacy. More particularly, the privacy engine is capable offunctioning on any of a number of different levels of security andprivacy, from the lowest point of communication (e.g., a firewall) tothe highest level (e.g., “who is my friend and who is not”). The userpreferences and profiles in the data store, then, can control operationof the privacy engine. For example, the privacy engine can retrieve,from the data store, a list of known friends such that, when the mobileterminal 10 is queried for an action to be performed in accordance withembodiments of the present invention, the privacy engine can indicatewhether the action is allowed by comparing the querying entity to thelist of friends. In one advantageous embodiment, the privacy enginefunctions based on devices and/or user applications operating on contextatoms in accordance with embodiments of the present invention, as wellas a type of action, e.g., put, get, subscribe, notify, etc., asdescribed below. It should be understood, however, that the privacyengine can also utilize other information, including other context atomsand data in the data store to make decisions as to permissions.

In addition to the sources of context-related information within themobile terminal 10 (e.g., sensors 100 and other data measuringelements), the context exchange architecture can provide for receivingcontext-related information from sources outside the mobile terminal. Inthis regard, the context exchange architecture includes a communicationsmanager 120, which can identify other context sources located within acommunication distance from the mobile terminal. In addition toidentifying the other context sources, the communications manager caninitiate contact with such context sources and, in general, providehousekeeping for communications with such context sources. Thecommunications manager can also function as a firewall (along with theprivacy engine 118). Advantageously, the communications manager canabstract different connection types to the context engine 102 such thatcontext atoms within the context engine can be stored and/or retrievedregardless of whether the communications manager communicates with othercontext sources via Bluetooth, GPRS, RFID, USB, or some other link type.

To facilitate the mobile terminal 10 communicating with the cellularnetwork, data network and/or other electronic devices, the contextexchange architecture also includes a service interface 122. The serviceinterface operates in much the same manner as the context API 106 in thecontext engine 102. In this regard, the service interface can comprise,for example, an over-the-air interface, as well as the underlyingtransport protocols (e.g., Ethernet, GPRS, TCP/IP). In accordance withvarious embodiments of the present invention, the context exchangearchitecture also provides a context exchange protocol. Generally, thecontext exchange protocol comprises a standardized protocol that allowsdifferent entities to exchange context atoms in a secure and reliablefashion, regardless of underlying architecture or transmission channel.In this manner, for example, at least a portion of the context atomsstored in the blackboard 108 can be stored in a separate memory device,such as a memory device coupled to an origin server.

The context exchange architecture can additionally provide for creatingnew context atoms, as well as deleting and/or modifying existing contextatoms. In this regard, the context exchange architecture can include ascript engine 124, which can store rules and conditions. In oneadvantageous embodiment, the script engine is capable of receivingexternal rules according to the context exchange protocol, andthereafter storing the rules. The script engine can then examine therules, subscribe to the relevant context atoms (described below), andthen execute the rules according to conditions defined in the rules. Thescript engine allows complex rules to be executed for different events.As such, the script engine can reduce the need for application code, aswell as reduce bandwidth requirements. Further, whereas the subscriptionmanager typically only transmits notifications when a context atomchanges, the script engine can further specify requirements of theconditions to transmit a notification. For example, the script enginecan specify conditions such as “transmit a notification only when thevalue changes by more than 20%.” In one embodiment, for example, therules can comprise one or more Java classes that include executable codedefining the conditions of the rules, and/or extensible markup language(XML) scripts that include Boolean logic defining the conditions of therules.

Each context rule typically includes one or more conditions and one ormore actions to be performed when the conditions are satisfied. In thisregard, the conditions of the context rules are typically related to thecontext atoms, or more particularly, the values in the context atoms.The script engine 124 can execute any of a number of different contextrules that include any of a number of different actions. For example,the context rules can specify the setting of a value of a context atom,the transmission of a notification, and/or the transmission of a contextatom to another device and/or application. Context rules can beexchanged between the mobile terminal 10 and other devices, such aselectronic devices, in a number of different manners. In one embodiment,for example, context rules can be exchanged in accordance with thecontext exchange protocol, such as to add a new context rule, remove acontext rule and/or notify a device of an expired or otherwise disabledcontext rule. Alternatively, context rules can be created internallywithin the mobile terminal.

The elements of the architecture have been described independent of themobile terminal cell diagram of FIG. 2. It should be understood,however, that the architecture of embodiments of the present invention,such as are illustrated in FIG. 3, are capable of being performed by amobile terminal that includes the elements illustrated in FIG. 2. Inthis regard, at least a portion of the elements of the architecture arecapable of being embodied in corresponding elements illustrated in FIG.2. For example, the blackboard 108 and/or data store 116 are capable ofbeing embodied in memory such as SIM 96, volatile memory 98 and/ornon-volatile memory 100. Also, the functions of the context engine 102,script engine 124, privacy engine 118 and/or communications manager 120are capable of being performed by controller 64 and/or processor 66.Additionally, or alternatively, one or more of the elements of thearchitecture of embodiments of the present invention can be performedby, and/or mirrored by, other elements of the cellular network and/orthe data network of FIG. 1. For example, a copy of the contents of theblackboard in the mobile terminal can be maintained in an operator'sserver (e.g., origin server).

As indicated above, context atoms typically include at least a name anda value, and may also include metadata information such as, for example,its reliability, accuracy, source, time of creation, etc. In operation,the context atoms are typically created by entities called “contextproducers.” A context producer can comprise any of a number of differentinternal or external systems, devices and/or elements capable ofcreating a context atom, but in one typical embodiment, a contextproducer can comprise an internal sensor 94 and/or user application 112,and/or an external system, device and/or element (e.g., sensor or userapplication) capable of pushing a context atom to the mobile terminal,such as in accordance with the context exchange protocol. Upon creationand/or receipt by the mobile terminal, the context atoms can be storedin the blackboard 108. For example, a thermometer can measure theambient temperature of the environment and process the temperaturemeasurement into the context atom “environment.temperature=95° C.,”which can then be stored by the blackboard 108. Additionally, oralternatively, a temperature change over a period of time can bedetermined, such as by a user application 112, and stored by theblackboard. As another example, a sensor (input device) can retrieve asignal from local beacons that identify the location of the mobileterminal and process the signal into the context atom“location.logical=sauna,” which can thereafter be stored by theblackboard.

The mobile terminal 10 can receive measurements from any of a number ofdifferent types of local beacons capable of transmitting informationsuch as, for example, a Bluetooth device and/or a RFID tag. In thisregard, when the local beacon comprises a RFID tag, a RFID transceiver82 within the mobile terminal can receive the signal from the beacon byperforming a service discovery to determine the RFID tag identifiers inthe vicinity of the mobile terminal. The identifiers can then be storedby the blackboard as the context atoms “local RFID tag environment.”Another application, referred to as a context consumer and describedbelow, can then subscribe to the “local RFID tag environment,” tothereby receive periodic updates of the measurements received by theRFID transceiver 82. In this regard, the mobile terminal can monitor thearea proximate the mobile terminal for electronic devices, and requestlocation information from them, if available. The mobile terminal canthen store any available location information back on the blackboard108, where other applications (context consumers) can get theinformation. In the example given above, a RFID beacon can announce itslocation to be a sauna, so that mobile terminal stores the context atom“location.logical=sauna” in the blackboard.

The preceding example illustrated the mobile terminal 10 storing acontext atom identifying a logical location (i.e., sauna). It should beunderstood, however, context atoms can be created to identify any of anumber of other types of locations, such as an exact location (e.g., GPScoordinates and/or addresses) and/or a relative location (e.g., “next toK-Mart”). The use of the GPS sensor 92 can be advantageous for providinglocation information, particularly when the mobile terminal is locatedoutdoors. Even the absence of GPS location information can be useful,however, such as indicating a probability that the mobile terminal islocated indoors, where the GPS sensor may not be able to detect locationinformation.

Once stored within the blackboard 108, context atoms are typicallyretrieved by entities referred to as “context consumers.” Like thecontext producer, a context consumer can comprise any of a number ofdifferent internal or external systems, devices and/or elements capableof retrieving a context atom. In this regard, when a context consumerdesires to retrieve a context atom, the context consumer connects to themobile terminal 10, such as according to the context exchange protocolwhen the context consumer is external to the mobile terminal oraccording to the context API 106 when the context consumer is internalto the mobile terminal. The context consumer can thereafter transmit arequest to the context engine 102 identifying the context atom by name.For example, a request may read “Please give me a context atom called“environment.temperature,” where the phrase “environment.temperature”represents the ambient temperature of the environment around the mobileterminal. Additionally, or alternatively, a context consumer cantransmit a request identifying a context atom by the source of therespective context-related information, and/or a combination of thesource and the context atom name, thereby permitting a context consumerto limit its queries to a specific context source. For example, acontext consumer can transmit the request “Please give me all thecontext atoms provided by the temperature sensor in the mobileterminal.” In one typical instance, an application may desire to act asa preprocessor to a singular sensor such that the application does notwant to receive context atoms associated with other sensors providingthe same context-related information. This, of course, means that thename of a context atom may not be unique, but that a context atom may becharacterized by one or more other attributes, including the source of acontext atom.

In response to the request, the context engine 102 can then query theprivacy engine 118 as to whether the context consumer is allowed toretrieve the requested information. If the context consumer haspermission to retrieve the requested context atom, the context atom istransmitted to the context consumer. In this regard, if the context atomis requested by an application internal to the mobile terminal, such as,for example, user application 112 or any other application associatedwith processor 66, the context atom is typically transmitted by thecontext engine through the context API 106. If the context atom isrequested by an external application, however, the context atom istypically transmitted by the context API 106 to the communicationsmanager 120, which will then transmit the context atom over acommunications link in accordance with the context exchange protocol.

If the context consumer does not have permission to retrieve therequested context atom, however, the context atom is not transmitted tothe context consumer. In such instances, the context engine 102 cantransmit a message to the context consumer indicating that permission toretrieve the context atom was denied, if so desired. In case of arequest received from an external application or device in accordancewith the context exchange protocol, the request may be dropped silently,such as in the case of an error or a resend request. In other terms, thecontext engine can merely ignore the request, and therefore not transmita response to the context consumer.

In addition to retrieving context atoms, context consumers may also benotified when particular context atoms change, such as when particularcontext atoms are added to the blackboard 108, removed from theblackboard, and/or otherwise modified. In addition, the contextconsumers can also specify the extent of a change required before therespective context consumer is notified of the respective change. Toreceive notifications of changes to particular context atoms, contextconsumers may subscribe to particular context atoms by requesting to beplaced on a notification list for one or more context atoms. As such,when the content of the respective context atom(s) contents change orexpire, the context engine 102 can issue a notification to the contextconsumer. As will be appreciated, however, the context engine willtypically check with the privacy engine 118 to determine if the consumeris allowed to receive such notifications. Thereafter, the context enginecan keep the context consumer notified of any changes by transmitting anotification to the context consumer following each such change orexpiration of the respective context atoms. Whereas any of a number ofchanges can require notification, the context engine typically transmitsa notification in the event of the appearance and disappearance of anatom, change of an atom contents, and/or termination of thesubscription.

Once a context consumer retrieves one or more context atoms, the contextconsumer can utilize the context atoms to determine a context of themobile terminal 10 (or the user of the mobile terminal), such that themobile terminal, for example, can be operated based upon the determinedcontext. In this regard, the context consumer can determine a context ofthe mobile terminal, or more particularly the user of the mobileterminal, based upon requested context atoms, and thereafter transmitthe context back to the mobile terminal such that the mobile terminalcan perform one or more actions based upon the context. For example, thecontext information can be utilized to configure the mobile terminalbased upon the determined context, such as by changing the volume of themobile terminal according to ambient noise (physical context), changingmenus displayed by the mobile terminal according to whether a user iswith a digital tribe associated with the user (social context), and/orhighlighting email messages that have the same context as an emaildisplayed by the mobile terminal (subject context). For more informationon the context exchange architecture, see U.S. Provisional PatentApplication No. 60/460,316, entitled “System, Mobile Station and Methodfor Managing Context-Related Information” filed on Apr. 3, 2003, thecontents of which are hereby incorporated by reference in its entirety.

In accordance with an embodiment of the present invention FIGS. 4-6provide a screenshot, otherwise referred to as a view, that is generatedby a computer program product, otherwise referred to as an application.FIG. 4 provides an example of an overview context view 200 incombination with a timeline view 300. In the context view historicalcontext information is presented in a matrix having fixed width timeperiod columns 210. The time span of the matrix, i.e., the width of thetime period column, may be configured by the user. Alternatively, thewidth of the time period column may vary according to the amount ofcontext data. Each row 220 within the matrix of the context viewrepresents a specific context category. In FIG. 4, five rows in theleftmost column are evident signifying five context categories (Context1-Context 5). Vertical scrolling of the view may provide for additionalcontext items to be displayed within the view.

The context view can be customized for the display of multiple contextcategories, i.e., context parameters. The context parameters mayinclude, but are not limited to, terminal movement, terminal location,detected devices in the proximate area of the terminal, detected personsin the proximate area of the terminal, files created by the terminal,files received by the terminal, calendar entries, temperature and/orother environment related parameters (e.g. barometric pressure,altitude, etc), health related parameters (e.g. blood pressure, bloodsugar, heart beat, etc) and the like. It should be noted that in certaininstances detected devices, persons, locations, etc, will not have oroffer exact identification information. In these instances, in order forthe context diary to provide specific context information related to thedevice, person, location etc., the user may be provided the option ofmanually editing or adding to the context information. However, in mostinstances, precise identification of the device, person, location etc.is provided by the context source and, as such, manual functionsimplemented by the user are limited.

Depending on the characteristic of the parameter, the visual attributethat appears within a cell of the matrix will vary and can be adjustedby the application user so that it is easier for the device user todiscern important aspects of the context view and patterns within atimeframe. It is noted that the FIG. 4 embodiment shows no visibleattributes within the cells of the matrix. The visual attribute mayinclude but is not limited to icons, such as flags, color-coding of thecell or the icons within the cell, varying the size of the icons withina column to signify a range or quantity and the like. Additionally, thevisual attribute may include number and/or text strings, lines, curves,bars, spots, topographic presentation or any combination of the abovementioned.

The context diary application will readily provide for the sharing ofhistorical context, content and/or setting information with otherdigital devices/users. In short-range communication environments sharingof the data may occur without the knowledge of the device that issharing the information. Therefore, the context diary application mayprovide for the labeling of information as private or public orotherwise prioritizing the data to insure that certain contextinformation is not readily shared with the public. In the FIG. 4 exampleeach column 210 represents an hour time period between 7:00 and 18:00within one of three days (Tuesday-Thursday, June 18-20). The hour timeperiod is designated by the corresponding number in the last entry ineach column. Alternatively, the time period in the columns 210 may beselected to be any time period, such as second, minute, day, week,month, year, etc. The application may be configured or the user maydesignate the time periods that are preferred for display within theview. Typically not all hours within a day will be displayed within theview because of display size limitations. In the illustrated embodimentthe 7:00-18:00 hour time periods signify the time periods in which thedevice is mostly likely to encounter or experience a change in contextinformation (i.e., the active hours). In addition, the application mayprovide for horizontal scrolling of the time periods within a day. Inaddition, it may be possible for the user of the device to configure thecontext diary application by providing a name, note bookmark, etc. for aspecific period of time. The application will then present the timeperiod identifying name, note, bookmark, etc. in the context view toprovide the time period specific meaning and to aid the user in findingthe time period within the historical context information database.

The timeline view 300 includes a time bar 310 and a media handle 320.The media handle will be moveable on the display for the purpose ofproviding viewable access to different time periods within the contextdiary. The media handle 320 depicted in FIG. 4 is shown in a stationaryrest position, preferably in a centerline position. In the centerlineposition the media handle is associated with an underlying column of thecontext view and the media handle will display the date associated withthe underlying column. Additionally, in the centerline position themedia handle will typically be associated with an underlying time unit,such as a day, of the time bar. The media handle will typically displaythe time unit, such as the date, associated with the underlying timeunit in the media handle. For more information related to the mediahandle, please refer to U.S. patent application Ser. No. 10/738,006,entitled, “Time Handle in a Media Diary Application for Accessing MediaFiles” filed on 17 Dec. 2003, in the name of inventors Finke-Anlauf etal. and assigned to the same inventive entity as the present invention.That application is herein incorporated by reference as if setforthfully herein.

The time bar may be hierarchal in nature such that it is presented tothe user with various levels that allow the user to maneuver through thelevels to pinpoint the specific time period associated with the contextinformation of interest. For example the time bar may incorporate levelsassociated with years, months, weeks, and/or days. The user of this timebar can start at the year level and maneuver through the month and weeklevel to locate the specific date associated with the media file. Formore information related to the time bar, please refer to U.S. patentapplication Ser. No. 10/715,162, entitled, “Time Bar Navigation in aMedia Diary Application”, filed on 17 Nov. 2003, in the name ofinventors Metsätähti et al. and assigned to same inventive entity as thepresent invention. That application is herein incorporated by referenceas if setforth fully herein.

FIG. 5 provides a combined context, media and timeline view, inaccordance with an embodiment of the present invention. The timelineview 200 is presented at the top portion of the view, the media view 400is presented in the middle portion of the view and the context view ispresented in the bottom portion of the view.

The media view 400 will typically display icons (not shown in FIG. 5)representing media files, such as text files, i.e., emails, SMS messagesor the like, image files, video files and the like. Additionally theicon may be presented with a file identifier in the form of a portion offile related metadata. For a more comprehensive discussion regarding themedia view and the related media diary application, see U.S. patentapplication Ser. No. 10/792,175, entitled, “Media Diary IncorporatingMedia and Timeline Views”, filed on 3 Mar. 2004, in the name ofinventors Cartier et al. and assigned to the same inventive entity asthe present invention, That application is herein incorporated byreference as if set forth fully herein.

The media view portion of the diary will typically calendar events andprovide metadata for the events that will be used for future referencein the historical data presented in the context view. However, manyevents occur that are not planned and furthermore, are not provided foras an entry in the calendar/event log or otherwise displayed in themedia view. Therefore, the application of the present invention usescontext information to automatically interpret a certain time period asa period denoting an “event” and the diary application will create anevent entry in the context view of the application. The application willtypically provide for the user to personalize the automatically createdevent entry by adding text notes, title, images or a personalized icon.Once the application has identified a time period as an event period,the event can be utilized to provide display of all the contextinformation related to the event.

In the illustrated example of FIG. 5 the width of the time periods mayvary based upon the existence and density of historical contextinformation and/or the existence and density of media files for theparticular date. In the illustrated embodiment the width of the datecolumns 230 vary according to the volume of context or content. Forexample, the width of the column for Monday 17 June is condensed due toa small quantity of context information and/or media files. The date ofTuesday 18 June is also condensed by wider in column size due to thepresence of more context information and/or content information than ispresent on Monday 17 June. The date of Thursday 20 June provides for thewidest width column, a full mode column, corresponding to the presenceof the most context information and/or media files.

FIG. 6 provides a combined context, media and timeline view, inaccordance with an embodiment of the present invention. In the FIG. 6embodiment the width of the date columns 230 are fixed and, therefore,unlike the FIG. 5 embodiment, do not vary according to density ofcontext information and/or media files present on a particular date. Thewidth of the columns in the FIG. 6 embodiment, similar to the selectionof width in the FIG. 4 embodiment, is based upon a selected ordetermined time span of the context or media view.

Content presented in the media view, such as media files, SMS or MMSmessages, emails or the like may be associated, i.e., related to,context information presented in the context view and vice versa. When auser of the application selects a media item in the media viewassociated context information may be highlighted or otherwise indicatedin the context view. Additionally, associated context information may bepresented in an additional window that is opened and placed adjacent tothe selected and associated media item. From the alternate perspective,when a user selects one or more context items in the context matrixassociated media items in the media view may be highlighted, otherwiseindicated, presented, etc. Additionally, associated media item(s) may bepresented in an additional window that is opened and placed adjacent tothe selected and associated context information. Alternatively, contextinformation associated with one or more media items may be presented asa watermark or bookmark on the presentation of the of the media item.

Additionally, when a media item is received, created or otherwise editedwithin the media diary portion of the combined application contextinformation related to the moment of receipt, creation, editing, etc,may be added to the media file in the form of metadata. For example,when a multimedia presentation is presented at a meeting, contextinformation related to the meeting (e.g. time of the meeting, devicesencountered, environment conditions, persons encountered, etc) is addedto the presentation's metadata. Similarly, when the presentation iselectronically communicated to meeting participants context informationrelated to the communication is added as metadata to the presentationfile. The media file's context information is accessible in the contextdiary application according to the time period of the presentation orthe time period of the electronic communication.

FIG. 7 provides an alternate embodiment of the invention, in which thecontext view is displayed and the time periods are defined by auser-selected context category. For example in the illustratedembodiment the user has selected device location as the context categoryof interest. This category is presented in bold faced type along the toprow of the presented context categories. The time periods in thetimeline view are defined in relation to the user-selected contextcategory, such as movement of the device from location to location. Forexample, during the first time period from 00:00 to 7:28 the device islocated at the user's residence. This time period is indicated in theleftmost column of the presentation. During the second time period,defined from 7:28 to 7:55, the device is located in the user'sautomobile and during the third time period, defined from 7:55 to 9:30,the device is located at the user's place of employment, in the user'soffice. The context information for the other categories in thepresentation relate to the time periods that are defined by the selectedcategory. For example, during the initial period when the device islocated at the user's residence, the movement category indicates thatthe device is stable, the temperature reflects the average temperatureat the residence, the device category indicates that no othershort-range wireless devices were encountered, the persons met categoryindicated no persons were met or available, the user's blood pressurewas determined to be normal and no media items, available in the mediaview, were captured or received during this period. As the device anduser move to the car, i.e. the second time period, the movement categoryindicates device movement, the temperature reflects the averagetemperature in the automobile, the device category indicates that aBluetooth® headset and a music player were encountered, the persons metcategory indicated no persons were met or available, the user's bloodpressure was determined to be normal and no media items, available inthe media view, were captured or received during this period.Additionally, one of the context categories may describe actions and/ormanipulations performed on media files, for example from 7:56-12:08 onecell of the column may describe “7 emails received: 1 SMS received:Presentation for monthly meeting ppt sent to John Doe”.

FIG. 8 provides an alternate embodiment of the context view 500 of thecontext diary application, in accordance with the present invention. InFIG. 8, context information is presented in a day view that isscrollable vertically and horizontally. Selection of the day may beaccomplished in a calendar view or by format. Scrolling from day-to-daymay also occur by vertical scrolling. In the FIG. 8 embodiment the cellsof the matrix include icons that represent context information. Thecontext view can be customized for the display of multiple contextitems/parameters. Depending on the characteristic of the parameter andthe importance to the device user, the visual attribute or iconassociated with the parameter can be adjusted so that it is easier forthe user to discern. Furthermore the visual attributes or icons can workin unison to support the functionality and readability of theinformation, aiding the device user in personalizing the contextinformation that is presented.

The left-hand-most column 510 lists the time periods in one hour cellsof time. Context items are listed in the event column 512, the locationcolumn 514, the Bluetooth® device column 516, the device movement column518, the temperature column 520 and the content column 522.

The event column 512 displays event icons 524. In the illustratedembodiment the event icon is a meeting icon indicating that thedevice/use attended a meeting in the 16:00 hour. The straight line icon526 in the 17:00 hour cell of the event column indicates that themeeting continued through the 17:00 hour.

The location column 514 may display straight line icons 526, zigzag lineicons 528, and color-coded flag icons 530. Straight line icons indicatethat the device stayed in the same location as the previous hour. Zigzagline icons indicate that the device was located in different locationsthroughout the time period. A flag icon indicates that the device stayedin the same location for the time period and the color of the flag mayindicate whether the location is labeled or unlabeled.

The Bluetooth® device column 516 may display color-coded blob icons 532.The size of the blob indicates a quantity range of Bluetooth® devicesencountered during the time period. The movement column 518 may alsodisplay color-coded blob icons 532. The size of the blob indicates theamount of movement encountered by the device during the time period. Inthe above mentioned cases, each color-code or icon type indicates adifferent type of device or movements indicated. The temperature column520 may include color-coded highlighting 534. Each color-code indicatesa specific temperature range.

The content column 522 may display color-coded blob icons 532,color-coded bar icons 536 or the like. Each color-code or icon typeindicated a different type of media file; i.e., image file, video file,email file, SMS file, etc. The size of the icon will typically indicatea range of quantity of files that have been captured, received,presented, created edited, sent etc. during the time period.Additionally, cells in any of the context columns may include one ormore color-coded indicators.

The number of context columns presented will typically be limited due todisplay size. Optionally the context columns may be scrollable toprovide for the presentation of additional context categories. The userof the device may configure the context columns so that the mostrelevant context categories are presented in the view are presented inthe initial view, i.e., prior to scrolling.

A device user may select and activate either a row, a column or a cellwithin the context view matrix to provide more detailed informationrelated to the context category, the time period or the context item fora given time period. The more detailed information will typically bepresented in a separate view on the device display on a time-line formatwith context information, a time-line format with content information orin a content presentation format. Depending on the attributes of thecontext category, item or period of time, the more detailed informationcan be designed to allow the user to customize the labels of therecognized content value. Further selection and activation within themore detailed view may be possible to provide the user with even moreinformation related to context category or context item.

In addition to storing historical context data the context diary mayprovide for presentation of concurrent context information; i.e.,information related to the current state of the user/device. In oneembodiment of the invention the digital device will monitor the currentenvironment for context information. Monitoring may involve receivingcurrent context information from the device itself; i.e., internalsensors or applications or receiving current context information fromexternal sources via wireless communication. The current contextinformation can be presented within the context view, along with thehistorical context information or the current context information may bepresented in a separate view. For example, the current contextinformation may be presented in an idle screen view (i.e., when thedevice is idle), a ticker bar view that scrolls continuously the currentcontext information, on a background image (i.e., wallpaper image), on aspecial application or the like. The current context information will beupdated within the view when a change in context occurs. For example,movement of the device to a new location, detection of a new contextinformation source, disappearance of a context information source orwhen detecting a new value for at least one of the currently identifiedcontext items. Additionally, the context diary application may providefor further access to a current context item by selecting a context itemthrough appropriately configured user-interface activation. Furtheraccess of the current context item may entail accessing or communicatingwith the source of the current context item or changing a currentsetting value of the context item.

FIG. 9 provides an example of an idle mode view 600 that presents theuser of the device current context information when the device residesin the idle mode state, in accordance with an embodiment of the presentinvention. The idle mode view may display selectable icons or othergraphical indicators to context items that are specific to the currentlocation or moment in time. For example, a flag icon 610 indicateslocation information, a folder icon 620 indicates a calendar entry, akey button icon 630 indicates device profile information, a temperatureicon 640 indicates current temperature, a device icon 650 indicatesdevices available in the location, person images 660 indicateindividuals available at the current time and location and an avatar 670indicates the device user and the current condition/movements of thedevice user. These icons may be selected by the device user to providefurther information related to the context and/or access to the contextsource.

FIG. 10 provides an example of a rolling ticker bar window 680 thatpresents the user of the device current context information in acondensed format, in accordance with an embodiment of the presentinvention. The rolling ticker bar window allows current contextinformation to be displayed in condensed form while the remainder of thedisplay screen 690 is free to provide presentation of anotherapplication UI, an idle screen, background image or the like. Rollingticker bar window will display a continuous roll of selectable icons orother graphical indicators related to context items that are specific tothe current location or moment in time. As shown, a flag icon 610indicates location information, a folder icon 620 indicates a calendarentry, a key button icon 630 indicates device profile information, andperson images 660 indicate individuals available at the current time andlocation. Additional icons representing additional context items may bepresented as the window scrolls.

Additionally, the context diary application of the present invention isnot limited to display of historical or current context information andmay, in certain instances, provide for future context information orforecasts of future context information in a future context view. Suchfuture context information may include, but is not limited to, calendarevents, weather forecasts, electronic program or service guide (ESG orEPG) information or the like. The future or forecast context informationmay be presented also in the historical context view as separatecategories of future information, forecasted information,realized/actual information and the difference between forecasted andactual information.

In accordance with another embodiment of the present invention, thecontext diary application may provide for the creation of multimediafiles that collect context items and provide a presentation on contentand context related to a specific period of time, event, and location.In creating the multimedia presentation the application will rely onfiles associated with a time period, files associated with a location,avatars selected to represent movement, recognized local devices and thelike. The multimedia presentation will typically be created in a formatsupported by the digital device to provide for local sharing or MMScommunication of the presentation.

FIGS. 11A-11G provide an example of the creation of a multimedia file;in accordance with an embodiment of the present invention. In FIG. 11A,the user activates the “select” function 700 and is presented drop downwindow 710. The user then selects the “create context movie” option 712from the drop down window. FIG. 11B provides a context movie view 720that is presented upon activation of the “create context movie option712. The context movie view provides the device user with options forcreating the movie including content files and context information on aspecific time span. Content files are presented chronologically withcontext information. For example, time span 722, movie length 724 andbackground music 726 may be selected by the user of the device.Selection of the time span will activate the calendar view 730, shown inFIG. 11C. The user will then select appropriate dates and or times fromthe calendar view and the movie/presentation will incorporate allmultimedia files related to the chosen dates/times. Selection of themovie length will activate the ‘select movie length” view 740, shown inFIG. 11D. Selection of a movie length will define the length of the clipthat is to be created. Alternatively, the length of the clip may beautomatically determined by the amount of context and contentinformation in the selected time span. Selection of the background musicwill activate the “music selection” view 750, shown in FIG. 11E.Selection of a music clip will result in the music clip being used asbackground music in the presentation. Alternatively, the music clip oran audio file may be selected according to context informationdescribing a music clip or the audio file that was presented at thespecific moment or period of time.

Once all the movie options have been defined the user will be presentedwith a preview view 760, as shown in FIG. 11F. The preview view allowsthe user to preview the created video presentation and to subsequentlysave the video through the options menu, if the user desires to savesuch. Additionally, the options menu will provide an option for the userto send the file, via Multimedia Message Service (MMS) to one morerecipients. FIG. 11G provides an MMS view 770 that attaches the createdvideo file and allows the user to define the recipients 772 and add textto the message 774.

Thus, the present invention provides for applications, devices andmethods for providing contextual information to digital device user in apresentation that associates the contextual information with a period oftime. As such the present invention provides for efficient user accessto historical, as well as, current contextual information. In addition,the present invention provides for current contextual information to bepresented in an idle mode view, a scrolling bar mode view or anotherinconspicuous view. Additionally the present invention provides a novelmeans of using context information to create multimedia filepresentations; image or video file presentations that can be associatedwith previous periods of time, locations or other context data.

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the inventions are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

That which is claimed:
 1. A mobile terminal comprising: two or moresensors for sensing two or more types of context data; a sensorapplication program interface (API) for processing the sensed data toprovide one or more context atoms relating to specific pieces ofcontext-related information, wherein each context atom comprises anentity having an associated name, value, source, and date; a contextengine for receiving and storing the context atoms, the context engineincluding a context silo for storing the context atoms and a context APIfor allowing one or more user applications to access the context data; adata store coupled to the context engine for storing information relatedto a user; a privacy engine for providing security; a communicationsmanager for identifying context sources external to the mobile terminaland initiating contact with the external context sources; and a displayfor presenting the context-related information and/or information fromthe external context sources in the manner of a diary, wherein thecontext silo further comprises a subscription manager for providingsubscriptions to one or more of the context atoms and for providingnotifications in the event of a change in the context atoms.
 2. Themobile terminal of claim 1, wherein the subscription managerperiodically polls the context atoms based upon the subscription andnotifications.
 3. The mobile terminal of claim 1, wherein the one ormore context atoms further includes attributes including metadatainformation, the metadata information associated with reliability of thecontext-related information.
 4. The mobile terminal of claim 3, whereinthe metadata information further includes information associated withaccuracy, source, and time of creation.
 5. The mobile terminal of claim1, wherein operation of the privacy engine is responsive to theinformation related to the user.
 6. The mobile terminal of claim 5,wherein the information related to the user comprises user preferences,profile information, and/or permissions for the privacy engine.
 7. Themobile terminal of claim 1, further comprising a time-line based userinterface, wherein the time-line based user interface is used to displaythe context information in relation to a period of time.
 8. A methodcomprising: sensing, in a mobile terminal, two or more types of contextdata; processing the sensed data to provide one or more context atomsrelating to specific pieces of context-related information, wherein eachcontext atom comprises an entity having an associated name, value,source, and date; receiving and storing the context atoms in a contextengine; storing, in a data store, information related to a user;providing different levels of security based on the information relatedto the user; identifying context sources external to the mobile terminaland initiating contact with the external context sources; using thecontext-related information and external context sources to provide apresentation on the mobile terminal of multimedia files; and providingsubscriptions to one or more of the context atoms and for providingnotifications in the event of a change in the context atoms.
 9. Themethod of claim 8, wherein the one or more context atoms furtherincludes attributes including metadata information, the metadatainformation associated with reliability of the context-relatedinformation.
 10. The method of claim 9, wherein the metadata informationfurther includes information associated with accuracy, source, and timeof creation.
 11. The method of claim 8, wherein operation of the privacyengine is responsive to the information related to the user.
 12. Themethod of claim 11, wherein the information related to the usercomprises user preferences, profile information, and/or permissions forthe privacy engine.
 13. The method of claim 8, further comprising:providing time-line information; and displaying the context informationin relation to a period of time.
 14. A mobile terminal comprising: twoor more sensors for sensing two or more types of context data; a sensorapplication program interface (API) for processing the sensed data toprovide one or more context atoms relating to specific pieces ofcontext-related information, wherein each context atom comprises anentity having an associated name, value, source, and date; a contextengine for receiving and storing the context atoms, the context engineincluding a context silo for storing the context atoms and a context APIfor allowing one or more user applications to access the context data; adata store coupled to the context engine for storing information relatedto a user; a privacy engine for providing security; a communicationsmanager for identifying context sources external to the mobile terminaland initiating contact with the external context sources; and a displayfor presenting the context-related information and/or information fromthe external context sources in the manner of a diary.
 15. A methodcomprising: monitoring the external environment of a mobile terminal todetermine two or more types of context data, wherein the monitoringincludes sensing two or more types of context data from sensors disposedwithin the mobile terminal; presenting a context view that provides atleast two context categories respectively associated with the two ormore types of context data and at least one context item within each ofthe at least two context categories, wherein each of the at least onecontext items is related to a period of time; presenting a calendarview; and receiving selection of a period of time within the calendarview for presenting the context view that is associated with the periodof time so as to present a context diary.
 16. The method of claim 15,wherein the presenting a context view that provides at least two contextcategories associated with the two or more types of context data and atleast one context item within each of the at least two contextcategories further comprises presenting the context view in combinationwith a timeline view that provides at least two context categoriesrespectively associated with the two or more types of context data andat least one context item within each of the at least two contextcategories associated with a period of time within the timeline view.17. The method of claim 15, wherein the presenting a context view thatprovides at least two context categories respectively associated withthe two or more types of context data and at least one context itemwithin each of the at least two context categories further defines theat least two context categories as chosen from the group consisting ofterminal location, short-range communication devices encountered,location characteristics, and terminal movement.
 18. A methodcomprising: monitoring the external environment of a mobile terminal todetermine two or more types of context data, wherein the monitoringincludes sensing two or more types of context data from sensors disposedwithin the mobile terminal; presenting the current monitored contextdata in user-accessible form on a display of the mobile terminal; andupdating the context data on the display when there is a change in themobile terminal environment; wherein the updating further defines achange in the mobile terminal environment as chosen from the groupconsisting of detection of an external context data source; andacknowledging the disappearance of a signal from a detected externalcontext data source.
 19. The method of claim 18, wherein the monitoringthe external mobile terminal environment to determine two or more typesof context data further includes receiving one or more context data froman external context data source.
 20. The method of claim 18, wherein thepresenting the monitored context data in user-accessible form on adisplay of the mobile terminal further comprises presenting themonitored context data on an idle screen mode.
 21. The method of claim18, wherein the presenting the monitored context data in user-accessibleform on a display of the mobile terminal further comprises presentingthe monitored context data on a ticker bar mode that scrolls monitoredcontext data.
 22. The method of claim 18, wherein the presenting themonitored context data in user-accessible form on a display of themobile terminal further comprises presenting the monitored context dataon a background image.
 23. The method of claim 18, wherein thepresenting the monitored context data in user-accessible form on adisplay of the mobile terminal further comprises presenting themonitored context data within a secondary application.
 24. The method ofclaim 18, further comprising accessing the monitored context data toprovide user management of the context data, wherein the accessing themonitored context data to provide user management of the context datafurther comprises accessing an external device that is providing one ormore context data.
 25. The method of claim 18, further comprisingaccessing the monitored context data to provide user management of thecontext data, wherein the accessing the monitored context data toprovide user management of the context data further comprises accessingthe monitored context data to change a parameter related to the two ormore context data.